Skip to content

Use DeepSeek agent for BuffBench - #622

Open
jahooma wants to merge 2 commits into
mainfrom
jahooma/buffbench-base2-deepseek
Open

Use DeepSeek agent for BuffBench#622
jahooma wants to merge 2 commits into
mainfrom
jahooma/buffbench-base2-deepseek

Conversation

@jahooma

@jahooma jahooma commented May 8, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Switch BuffBench main runner from base2-free-evals to base2-free-deepseek.

Validation

  • Reviewed git diff origin/main...HEAD.

@codebuff-team

Copy link
Copy Markdown
Contributor

The stated change — swapping base2-free-evals for base2-free-deepseek in evals/buffbench/main.ts — is a one-line, sensible change and clearly in scope.

However the diff is much larger than the description suggests. agents/base2/base-deep.ts and agents/base2/base2.ts also drop file-picker, file-picker-max, directory-lister, and glob-matcher from spawnableAgents, rewrite the system/step prompts to point at glob/list_directory/read_subtree/read_files instead, and get a bunch of unrelated prettier-style reformatting (multi-line function signatures, template literal indentation). None of that is mentioned in the PR body, and there's no rationale given for retiring those sub-agents or reason to believe it's tied to the DeepSeek swap.

As it stands this reads like two unrelated changes squashed together: a trivial eval config change, and a meaningful behavioral change to how base2/base-deep orchestrate sub-agents. The latter deserves its own PR with a description of why those agents are being removed and what testing (if any) shows the direct-tool approach performs as well. Please split these so each can be reviewed and ported on its own merits.

@codebuff-team codebuff-team added bot:triaged Classified by the community triage bot pr:needs-work Right idea, not mergeable as written labels Aug 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bot:triaged Classified by the community triage bot pr:needs-work Right idea, not mergeable as written

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants